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Abstract: Nose tip detection is an essential point in 3d face alignment, pose estimation and the discrimination process. A 
method to determine nose tip location based on exploiting local descriptor s( shape index) from each point cloud on the 3d 
surface mesh is presented in this paper. Furthermore, theeffective energy constraint was appliedover each 3d mesh data 
point. Then NN-Nearest Neighbour classification methodwas applied in order to locate the nose tip and nose region based 
3d feature extraction techniques. The detection method is also characterized by the refining of candidate points for nose tip 
regions and is robust against pose and expression variations. All computations were based on the pure 3D mesh data points 
of a face without texture information. The proposed method was tested using a 3d face image sample database FRAV3d, and 
GAVADB database. The experimental results show 96% and 76% correct nose tip detection for FRAV3d and GAVADB 
databases respectively. 

I. INTRODUCTION 

The detection of three dimensional facial landmarks is an important stage in biometric applications, computer vision 
and computer graphics. In order to perform face registration, recognition, facial expression recognition, facial shape analysis 
and segmentation in 3d face recognition systems, the facial landmarks must be located. Compared to other facial landmarks 
the nose tipis one of the most important and critical landmarks because of its distinct shape and symmetrical property. 
Technically in a 3D face environment, the nose tip is the point with the maximum depth value. In reality, the nose tip point 
(region) has a maximum depth value which is true only in the frontal image[l]. 

A lot of researches have been done to locate the nose tip of 3d faces. Breitenstein et al. [12] proposed an approach 
to detect an invariant pose for the nose tip. For each point in the acquisitioned image, the shape signature was calculated and 
the corresponding pose hypotheses were generated in parallel. Then, the nose tip was selected using error function which 
compares the input range image to the pre-computed pose images of an average face model. 

Zhu et al.[10] assumed the protuberant points as nose tip candidates. Since only frontal faces were considered, symmetry 
calculation was performed in order to obtain the nose tip location, which was implemented using the direction comparison of 
normal vectors. 

Werghi et al.[9] investigated a method of nose tip detection and frontal face extraction by estimating mesh quality. 
They measured and assessed mesh surface quality by extracting groups of triangular facets. The nose tip was detected by 
identifying the single triangular facet using a cascaded filtering framework mostly based on simple topological rules. 
Bagchi et al. [8] used HK classification to detect the nose tip and eye corner which is based on curvature analysis of the 
entire facial surface. 

In this paper,a nose tip detection approach is presented for 3d facial data points. This approach is based on a 
simultaneous filtering technique. The extraction of local geometry information from the 3d data point is carried out, at the 
same time that the effective energy constraint is computed for nose tip candidates. Therefore, the two region points which are 
deduced from the two approaches feed into NN-Nearest Neighbour classifier to obtain the nose tip location on the 3d facial 
image. 

II. 3D SHAPE DESCRIPTORS 

Surface curvature is a property of the local surface, which has the advantage of being invariant against rigid 
transformation (translations and rotations) [15]. Therefore the shape descriptors derived from curvature have been adopted in 
this work. Curvature based measures, which are related to second-order derivatives of theraw depth measurements, were used 
to extract features from 3D facial images. To understand principal curvatures, imagine the normal on a surface and an 
infinite set of planes (a pencil of planes) each of which contains this normal. Each of these planes intersects the surface in a 
plane curve and the principal curvatures are defined as the maximum curvature K u and minimum curvature K 2 of this infinite 
set of plane curves. The directions that correspond to maximum and minimum curvatures are always perpendicular and are 
called the principal directions of the surface. Principal curvatures are in fact the eigenvalues of the Weingarten matrix, which 
is a 2 x 2 matrix containing the parameters of aquadratic local surface patch fitted in a local plane that is aligned to the 
surface tangentplane[4]. 

III. THE PROPOSED NOSE TIP DETECTION METHOD 

The proposednose tip detection method is based on the extraction of local descriptors and statistical characteristics 
from 3d data points. Figure (1) shows the outlines of the proposed method.The process starts with the acquisition of the 3d 
face data by a face scanner. In the next stage the3d data is filtered and smoothed by median noise removal. Preprocessed 3d 
data is fed into the next stage in order to perform the calculation of the local shape index and effective energy. At the final 
stage, the nose tip location and nose regionare extracted by applying the NN-Nearest Neighbour classification method on the 
two candidate regions. The details of these stages are given in the following sections. 
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Fig 1. Block Diagram of The Proposed nose tip detection method 



1. Pre-processing 

Preprocessing is an important stage of the recognitionsy stems, since all the features will be extracted from theoutput 
of this step [3]. The 3d face images are obtained by 3d scanner.Each captured image (mesh) is composed of 3d coordinate 
points, set of faces (polygons), normal of vertex, and may also contain texture information. Therefor the first step is to 
remove spikes formacquisition datausing filtering techniques. A 2d median filter is used to smooth 3d data which is applied 
to Z- values. This technique will remove spikes from the 3d facial image. 



2. Curvature Analysis 

The local curvature information about a point is independent of the coordinate system. The feature points and 
geometry information of the 3d surface is captured by calculatingthe shape index of each 3d point. Shape index is 
extensively used for 3D landmark detection [14] .And is a scale independent component derived from principle curvatures. In 
other words the shape index values represent continuous mapping of principal curvature values (K max , K min )of a 3D face 
pointPinto the interval [0,1], and can be computed as follows: 

_ 1 1 (J^max K m i n ) 
2 71 (J£ max Kmin ) 



The principle curvatures (K max , K min )are in fact the eigenvalues of the Weingarten matrix, which is a 2x2 matrix 
containing the parameters of a quadratic local surface patch [4]. The surface patch equation is: 

f(x, y) = z = ax 2 + by 2 + cxy + dx + ey + f (2) 

Since (x^y^Zi) are neighbouring points and a,b,c,d,e,f are thesurface parameters to be found by the least square 
fitting technique. The Weingarten matrixformula is: 




The shape index captures the curvature value for each point on the surface. Every distinct surface shape corresponds 
to aunique value of shape index, except the planar shape. Points on a planar surfacehave an indeterminate shape index, 
smccK max = K min = 0. In fact there are five well-known shape types and their locations on the shape index scale are shown 
in figure(2): 




Spherical cup=0.0 Rut =0.25 Saddle = 0.5Ridge =0.75 Spherical cap =1.0 

Fig 2.Sample representative shapes on the shape index scale[13] 

After calculating shape index values for all 3d facial data points, the surface points that have a shape index within a 
threshold range are extracted, which indicates the convex regions of thesurface of the face. The low values of the shape index 
represent a spherical cup while a high values representsa spherical cap. 

This step will reduce the search space for nose tip candidate regions and produce at the same time what is called candidate 
nose tip regions(l). 
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3. Effective Energy Calculations 

The usual method to find the nose tip candidate region on 3d face images is to select N points with highestZ- value 
from the entire 3d mesh data points [7]. This process will reduce the possible candidate nose tipregions 
significantly. However with pose variation this processwilllead to incorrect regionlocations.As a resultthe effective 
£ft£rgyconstraint which wasadopted in[2]was applied over 3d mesh data points in order to collect the points which satisfy the 
locally protuberant conditions and to locate convex regions in the facial image. First of all the neighbours for each vertex? in 
the polygons mesh should be found.This is done by finding the faces that share this vertex.The effective energydi for each 
neighbour points/ 3 ; is calculated using the following formula: 

d i = (P i -P)-N p = \\P i -P\\-cosG (4) 

Where 6 is the angle between (P t — P) vector and the normal vector Af p isthe normal vector of the vertex P, which 
is calculated according to the common method based on a weighted average of theadjacent faces normal n^for a face 
composed of vertices vl, v2 and v3.The face normal is given by the formula (5) which is given in [4]: 

(vl - v2) x (vl - v3) 

Uf = \\(vl - v2) x (vl - v3)\\ (5) 
Next the points that have a negative value of d t (peaked region)are selected, which leadsto a reduction in the 
searching space for the nose tip candidate points. The Effective Energy constraint removes most of the points which are 
unlikely to be the nose tip. 

3.1 Local characteristic 

There are severalpossible nose tip regionsdue to the fact that the other regions like (chine, cheeks, forehead, ...) 
have a negative effective energy also. Since the effective energy measure is based on distance and angle, a nose tip candidate 
vertex should have a small variance value (the amount of dispersion in relation to neighbouring vertices). The next step is to 
select the vertex with smaller variance value from nose tip candidate regions as shown in figure (3). The mean and variance 
for each candidate region is calculated as follows: 

n 

^^iTdt (6) 

i=l 

n 

o^ljidt-fi) 2 (7) 

i=l 

Where is the mean,cr 2 is the variance, n is the number of neighbouring vertex and d^is the effective energy for a neighbours. 




(b) 

Fig 3. Nose tip detection based on effective energy calculation a) for FRAV3D database, b)for GAVADB database 
4. Nose Tip Detection 
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The last step in the proposed method for locating nose tip is to apply the Nearest Neighbour algorithm on two 
candidate regions. The results shown inFigure (4).Figure (4-a) represent the result values that come from two 
techniquespresented, the green points represent the nose tip candidate which are extracted using the EE method, while the red 
points represent the nose tip candidate points which are extracted using the shape index descriptor. Figure (4-b)represents the 
result values after applying the NN-classifier. 




(a) Before applying NN-classifier 




(b) After applying NN-classifier. 



Fig 4. Presentation for values of two candidate regions (a) Before applying NN-classifier, (b) After applying NN-classifier. 

IV. DATABASE 

The proposed nose detection method wasappliedto two different 3d facial databases: the FRAV3D and GAVADB 
database [5] [6]. The FRAV3D database containeda 3D point cloud ofl06subjects,with 16 captures per person. In addition, 
each subject had a different pose (rotation about x-axis,y-axis,z-axis), 

which includes facialscans with frontal(l,2,3,4), 25° right turn in Y direction (5,6), 5° left turn in Y direction(7,8), severe 
right turn in Zdirection(9), small right turn in Z direction(lO), smilinggesture(ll), open mouth gesture(12), looking up turn 
in Xdirection(13), looking down turn in X direction(14), frontal 
images with uncontrolled illumination(15,16) as shown in [5] [11]. 

Furthermore the experimentshave been conducted over 40 random subjectsof the GAVADB database. Thiscontained 61 
individual(45 male and 16 female)including (9)a 3d faceimage for each individual, whereby each image consisted ofa 
3dmesh representing thefacial surface. This database did not contain normal of vertices, the normal for each face in a 3d face 
mesh was found in order to obtain the normal of verticesas described in section 2.3. 

The required programmes werewritten in Visual C++ platformwith assistedOpengl graphic library. 

V. EXPERIMENTAL RESULTS 
Experiment l:The first test was performed with the FRAV3D and GAVADB databases, which included locating the nose 
tip of the input 3d facial images using energy effective constraint for N- highest Z-values.The results shown in table (1) 
representthe detection rate for the nose tip in both databases. 

Experiment 2:The second test was performed with theGAVADB database,which included the following: 

• Locating nose tipfor the frontal 3d face image and profile sides withnon-acute rotation angles— 3 5° to + 35°around 
(x-axis, y-axis, z-axis) 

• Detecting the nose region for non-frontal 3d face images with acute rotation angles— 90°to + 90°around y-axis. 

The testwas performed based on EE constraint and a local shape descriptor for 3d data points. The results shown in table (2) 
(3) represent the detection rate for the nose tip and nose region respectively. 
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DB Name 


Nose tip detection rate 


FRAV3d 


96% 


GAVADB 


66.6 







Table 1 . Nose tip Detection rate based on EE constraint calculation 



DlT^^ 


Frontal and non frontal ] 


posses (— 35°to + 35°) 


EE constraint 


EE & shape index 


GAVADB 


66.6% 


96% 


Table 2. Nose tip Detection rate using EE constraint and Shape index for 


DB?Name^^ 


Profile Posses (-90°& + 90°) 


EE constraint 


EE & shape index 


GAVADB 


66.6% 


76.8% 



Table 3. Nose Region Detection 

VI. CONCLUSIONS 

In this paper, acombinedschemefor determining nose tip and nose region in a three dimensional facial image has 
been presented with the presence of expression and pose variations. The local shape information was exploited from each 
data point using ashape index descriptor.The vertices which have a shape index value within a threshold range were 
selected. This step produced the first candidate region of the nose tip. Furthermore the energy effective for each neighbouring 
vertex was calculated.Next, the candidate points that had negative EE and the smallest variance value were selected to 
collect the second candidate regions of the nose tip. 

The final step in the proposed method wasthe application of the Nearest Neighbour classification method on the two 
candidate regions. In this stepa detection process for nose tip location and nose region was performed with high rate detection 
for frontal and semi frontal posses. 
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